Three Perspectives on the Impact of Comparative Effectiveness Research on Decision Making

BaCkgRound: Comparative effectiveness research (CER) is rapidly adding to the amount of data available to health care coverage and payment decision makers. Medicare’s decisions have a large effect on coverage and reimbursement policies throughout the health insurance industry and will likely influence the entire u.S. health care system; thus, examining its role in integrating CER into policy is crucial. oBjECtIvES: to describe the potential benefits of CER to support payment and coverage decisions in the Medicare program, limitations on its use, the role of the Centers for Medicare & Medicaid Services (CMS) in improving the infrastructure for CER, and to discuss challenges that must be addressed to integrate CER into CMS’s decision-making process. SuMMaRy: a defining feature of CER is that it provides the type of evidence that will help decision makers, such as patients, clinicians, and payers, make more informed treatment and policy decisions. Because CMS is responsible for more than 47 million elderly and disabled beneficiaries, the way that Medicare uses CER has the potential to have a large impact on public and individual health. Currently many critical payment and coverage decisions within the Medicare program are made on the basis of poor-quality evidence, and CER has the potential to greatly improve the quality of decision making. despite common misconceptions, CMS is not prohibited by law from using CER apart from some reasonable limitations. CMS is, however, required to support the development of the CER infrastructure by making their data more readily available to researchers. While CER has substantial potential to improve the quality of the agency’s policy decisions, challenges remain to integrate CER into Medicare’s processes. these challenges include statutory ambiguities, lack of sufficient staff and internal resources to take advantage of CER, and the lack of an active voice in setting priorities for CER and study design. ConCLuSIon: although challenges exist, CER has the potential to greatly enhance CMS’s ability to make decisions regarding coverage and payment that will benefit both the agency and their patient population.


S17 Continuing Education Instructions
Content for this activity is based on the live symposium, "Three Perspectives on the Impact of Comparative Effectiveness Research on Decision Making," which was presented as a satellite symposium at the Academy of Managed Care Pharmacy (AMCP) 2011 Educational Conference on Thursday, October 20, 2011, in Atlanta, Georgia. This monograph and the live symposium were jointly sponsored by the Postgraduate Institute for Medicine (PIM) and StrataMed, LLC, and supported by an educational grant from Novo Nordisk Inc. There is no fee for this activity.

Target Audiences
This activity has been designed to meet the educational needs of physicians, pharmacists, and health plan decision makers involved in the design/conduct of comparative effectiveness research (CER), and those who are interested in the principles of CER and CER' s impact on government, industry, and managed care perspectives.

Statement of Need
Comparative effectiveness research (CER) is a growing element of U.S. health care reform. By comparing new drugs with established therapies (in addition to traditional placebo trials), CER may potentially improve the scientific basis for decisions about the appropriate treatment for patients. But CER' s growth also presents questions for health care decision makers. As the U.S. health care system evolves, many stakeholders must evaluate what CER means to them, how they can integrate CER to their advantage, how they can ensure that the CER being conducted is relevant to their needs and how their organizations need to transform to use that data effectively. From the perspective of the Centers for Medicare & Medicaid Services (CMS), and specifically Medicare, CER may greatly enrich decision making regarding coverage and payment that will benefit both the agency and their patient population. But challenges exist, including ambiguous legal limitations, insufficient staff and internal resources, and lack of CMS representation on federal governing boards.
From the pharmaceutical industry perspective, the proliferation of CER data may both positively and negatively impact many areas, such as who can access CER data, how communication about the data will be regulated, how policies can be made flexible enough to accommodate the data, and how CER may affect innovation.
From the managed care perspective, health care reform and the growth of CER means organizations must invest in better infrastructure and new understandings of a transforming market, changing customer bases, and evolving data.
As CER becomes an increasingly important aspect of the U.S. health care system, now is the time to begin consideration of its potential impact.

Introduction
Diana I. Brixner, RPh, PhD, and Gary Oderda, PharmD, MPH I n 2009, the Institute of Medicine (IOM) defined comparative effectiveness research (CER) as "the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care." 1 CER isn't restricted to just comparing drug A versus drug B-it's much broader than that. CER includes measuring interventions, approaches to care, and the delivery of care. The IOM also stated that "the purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels." 1 The IOM clearly identifies these 4 different stakeholders (consumers, clinicians, purchasers, and policy makers) for CER, and they look at the impact both on the individual and across populations.
But the IOM isn't the only organization defining CER. The Federal Coordinating Council for CER, established by the American Recovery and Reinvestment Act (ARRA) of 2009 to coordinate CER efforts, defines CER similarly, although it doesn't specifically mention delivery of care or overall populations, and it lists only 3 stakeholders: patients, providers, and decision makers (although we can probably assume that "decision makers" includes both purchasers and policy makers). 2 These are subtle differences, perhaps, but interesting to note. The related discussions around definitions of stakeholders have also contributed to ongoing political disagreements as to who is the audience of CER, and therefore who should fund it. 3 It is important to distinguish CER from patient-centered outcomes research. The Patient-Centered Outcomes Research Institute (PCORI), which replaced the Federal Coordinating Council in 2010, states that it will "commission research that is responsive to the values and interests of patients and will provide patients and their caregivers with reliable, evidencebased information for the health care choices they face." In addition, the PCORI says it is committed to a "rigorous stakeholder-driven process that emphasizes patient engagement." 4 So the research direction of PCORI appears to be more patient-oriented than either the IOM or Federal Coordinating Council definitions. However, beginning in 2013, funding for PCORI will begin to include $2-per-patient annual transfers from the Medicare Trust Fund and from private health plans. 5 With these major stakeholders (health plans) contributing so substantially to the funding, will the work that PCORI produces actually assist these health plans in making the informed decisions that they need?
All of these definitions of CER, along with their variations, slight or otherwise, affect many different stakeholder groups, who all have different needs and expectations. As the U.S. health care system evolves, each of these stakeholders will have to evaluate what CER means to them, how they can integrate CER to their advantage, how they can ensure that the CER being conducted is relevant to their needs, how they can address some of CER's limitations 6 and how they or their organizations will need to transform to more effectively use that data. Stakeholders will also need to evaluate how conducting and evaluating CER will fit with current Food and Drug Administration (FDA) regulations and guidance. 6 And methodologies and infrastructures for conducting effective and meaningful CER must be systematically assessed and enhanced, and such efforts will require considerable dialogue before new "best practices" can be established. 7 A 2010 article by Tunis discusses various aspects of CER infrastructure that will need to be addressed to assure such research will be designed, conducted and communicated with the greatest benefit to stakeholders. 7 The gaps in knowledge left by randomized control trials are an ongoing source of concern for physicians, patients, policy makers, and payers. Without information about how well a new medication works when compared to other existing alternatives, in a real world setting, stakeholders are sometimes left with a "trial and error" approach to decision making, which can result in expensive lessons. CER can, if effectively designed and conducted, help fill some of those gaps. Of course, the key word is "effectively." While CER itself is not new, the rapidly increasing demands of and expectations for CER will bring a host of implementation questions and issues. Integration of CER will not be simple or seamless, but the more dialogue we encourage now to identify and address some of those issues, the more effective the research will be. For example, in a 2010 commentary, Rubin warns against what he perceives as a tendency for researchers to use inadequate data sets for CER studies and recommends some ways to evaluate and select more relevant data sets. 8 In a 2011 article, Alemayehu and Cappelleri point out some of the historical weaknesses in conducting and reporting observational and non-randomized studies, including sources of bias, and recommend steps to minimize bias in the design, analysis and reporting stages of a study. 9 Without a doubt, the increase in CER will change the health care industry in nearly every area, from development costs to formulary decision making, from treatment decisions to product innovation. A key question for payers will be that of value. How does each stakeholder define the value of CER? How do they prioritize to get the most value from that research? It's recognized that the higher-quality, higher-priced products may in fact result in lower costs over time. But there must be evidence that demonstrates this for the purchaser. 10 And we must define parameters to measure success.
A global overview of CER was presented in the October 2010 issue of Health Affairs that complements many of the perspectives of the current report. In that issue, for example, Etheredge outlines the need for a high-performing CER system and a national database of CER studies. 11 Robinson proposes how public and private insurers may adapt their policies to incorporate CER. 12 Pearson and Bach propose a payment model for Medicare that uses CER to encourage Medicare to pay equally for comparable medicines. 13  W ith responsibility for more than 47 million elderly and disabled beneficiaries, 1 the way that Medicare uses comparative effectiveness research (CER) has the potential to have a large impact on public and individual health. Medicare's decisions also have a much larger effect on coverage and reimbursement policies throughout the health insurance industry, 2 and these effects will likely extend throughout the U.S. health care system. For this reason, it is critical to understand how Medicare might use CER. This article examines CER from the Medicare program's perspective, including the program's current use of CER, common misconceptions about the ability of the Centers for Medicare & Medicaid Services (CMS) to use CER, and the role CMS has in future CER development.

The Lack of CER Impedes Effective Decision Making
Medicare and, ultimately, every payer in the health care system have the same fundamental questions when it comes to making decisions about paying for the drugs, devices or other services they cover: "How does this technology work in my population under circumstances of normal practice, and does it provide meaningful health benefits compared with existing treatments over the long term?" The italicized terms represent the gaps in existing clinical evidence that CER is designed to fill. Without data that address these questions, payers (including CMS), must make decisions based on studies that often have used poorly validated surrogate endpoints, have been conducted in carefully selected populations and settings, or may include no relevant treatment comparators.
The quality of information CMS commonly uses for decision making is illustrated by a systematic review of the efficacy of offlabel use for 19 approved cancer drugs conducted by the Duke Evidence-based Practice Center, published in 2010. After reviewing several thousand trials, the authors concluded: "Because of the paucity of high-quality evidence, the data available-though voluminous-may have little meaning or value for informing clinical practice." 3 Without doubt, the paucity of high-quality evidence has direct ramifications on the performance of the Medicare program. The Looking at CER from Medicare's Perspective Penny Mohr, MA ABSTRACT BaCkgRound: Comparative effectiveness research (CER) is rapidly adding to the amount of data available to health care coverage and payment decision makers. Medicare's decisions have a large effect on coverage and reimbursement policies throughout the health insurance industry and will likely influence the entire u.S. health care system; thus, examining its role in integrating CER into policy is crucial. oBjECtIvES: to describe the potential benefits of CER to support payment and coverage decisions in the Medicare program, limitations on its use, the role of the Centers for Medicare & Medicaid Services (CMS) in improving the infrastructure for CER, and to discuss challenges that must be addressed to integrate CER into CMS's decision-making process.
SuMMaRy: a defining feature of CER is that it provides the type of evidence that will help decision makers, such as patients, clinicians, and payers, make more informed treatment and policy decisions. Because CMS is responsible for more than 47 million elderly and disabled beneficiaries, the way that Medicare uses CER has the potential to have a large impact on public and individual health. Currently many critical payment and coverage decisions within the Medicare program are made on the basis of poorquality evidence, and CER has the potential to greatly improve the quality of decision making. despite common misconceptions, CMS is not prohibited by law from using CER apart from some reasonable limitations. CMS is, however, required to support the development of the CER infrastructure by making their data more readily available to researchers. While CER has substantial potential to improve the quality of the agency's policy decisions, challenges remain to integrate CER into Medicare's processes. these challenges include statutory ambiguities, lack of sufficient staff and internal resources to take advantage of CER, and the lack of an active voice in setting priorities for CER and study design.
ConCLuSIon: although challenges exist, CER has the potential to greatly enhance CMS's ability to make decisions regarding coverage and payment that will benefit both the agency and their patient population. • Without comparative effectiveness research (CER), payers and Centers for Medicare & Medicaid Services (CMS) must make decisions based on the best evidence available, which often lacks headto-head data comparison, uses surrogate endpoints, and excludes the specific elderly or disabled population that Medicare covers. • There is a common misperception that the CMS is not allowed to use CER data when making decisions.

What is already known about this subject
• Contrary to a common misperception, CMS is allowed to use CER data when making coverage decisions, albeit with some limitations. • CER could be used when creating patient decision aids, establishing value-based insurance design, and in coding (determining whether there is enough of a significant therapeutic distinction for a particular product to assign it a new code). • CMS could use CER data in pricing decisions-specifically in the area of least-costly alternative (or reference pricing). area of decision making around prostate cancer treatments provides a good example. Prostate cancer is the leading cause of cancer and the second leading cause of mortality among men. 4 While there are many alternatives for treating prostate cancer, there is a lack of robust evidence about which treatments work best at what stage of the disease. 5 In 2002, CMS covered intensity modulated radiation therapy (IMRT) on the basis of limited information about its performance relative to brachytherapy (which was the standard in use at the time). At the time, the cost of IMRT was about double that of brachytherapy. 6 Within a few years, roughly one-third of beneficiaries were using IMRT, and expenditures were well over a billion dollars for that particular technology. 7 More recently, proton beam therapy was introduced, and Medicare contractors made the decision to cover it. This essentially doubled Medicare expenditures again for a course of radiation therapy for prostate cancer, but still without good evidence that it improves the quality of care over IMRT or even over brachytherapy. We still do not know which therapies are more effective for which patients and which stages of prostate cancer, but treatment costs for this condition continue to escalate. Unfortunately for prostate cancer patients, these types of decisions not only affect their out-of-pocket obligations, but also could result in exposing them to undue toxicities and risk.

What this article adds
A defining feature of CER that makes it fundamentally different from the way clinical research has been done in the past is its purpose. According to the Institute of Medicine (IOM), "The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels." 8 To this effect, CER provides unique opportunities for CMS to improve its policies to ensure medical services are used more appropriately.

Can CMS Legally Use CER?
There is a common misperception that CMS is not allowed to use CER results when making decisions. The truth is that CMS is allowed to use CER data, albeit with some limitations. The limitations imposed on CMS's use of CER are fairly modest. CMS must use an iterative and transparent process, and they cannot use CER as a sole source of information. 9 The Patient Protection and Affordable Care Act (ACA) 9 of 2010 includes specific limitations on Medicare's use of CER. 10 CMS can't use it in a manner that treats the life extension of the elderly, disabled, or terminally ill differently than others, and they can't use it to preclude or discourage choices between extending life or risk of disability. 10 Additionally, CMS can't include mandates for coverage or payment, cannot use cost per quality adjusted life years threshold to determine coverage or payment and must use evidence only as part of a larger process. 10 In fact, these limitations reflect the way CMS already makes coverage decisions, and these limitations do not fundamentally change CMS's current processes.
The larger question is how these limitations from the ACA affect CMS's ability to use costs. The Patient-Centered Outcomes Research Institute (PCORI), a public/private entity charged with identifying research priorities, establishing an agenda, and fund-Looking at CER from Medicare's Perspective ing future CER studies, created under the ACA, 9 cannot fund research or advocate for a threshold for quality-adjusted life years (QALYs). In addition, the ACA imposed a restriction that the Secretary of Health and Human Services, acting for the Medicare program, could not use a "threshold" for QALYs in their coverage determinations. CMS has gone on record in guidance documents stating that they don't use cost or cost-effectiveness in their decision making. This restriction against using a threshold of QALYs is not a new limitation for CMS, but it is now incorporated into statute.
Another important aspect of the ACA is that it explicitly allows CMS to use differential copayments based on cost, which could open the potential of using value-based insurance design within the CMS program.

CMS's Obligation to CER: Making Data Available for Research
In recent years, significant federal funds have been expended in building an infrastructure to support CER, such as registries and claims databases. One of the legislative requirements is that CMS must make its data available for PCORI-sponsored research. Although CMS data have been available for a long time, the data have been relatively difficult to obtain. This new requirement may facilitate researcher access.
To that end, one of the major investments has been the development of the Multi-Payer Claims Database (MPCD). 11 The database combines Medicare data, including the institutional claims in Part A, outpatient claims from Part B, and the drug data from Part D, with claims data from private insurers. This potentially will deliver a much more robust observational data set for doing CER. Currently, 1 private payer is contributing claims data and hopefully more insurers will contribute data over time. Medicare data provide a rich resource for CER.
Using existing Medicare data, as with other claims data, is challenging. It is not possible to determine factors that contribute to treatment decisions, patients are not randomly assigned to treatment, and selection bias is an issue. These could be resolved if beneficiaries were randomly assigned to plans with different formularies based on factors such as low income subsidies. 12 However, much work remains in determining how to implement this requirement. Some of the outstanding questions include: How should a public/private partnership like this be sustained? Should there be benefits to the participants who add their private claims to the database, and if so, what type of benefits? Who will own the data? And how should the data be made available to researchers? Another important question is whether these data will be linkable to clinically rich information from randomized controlled trials or electronic health records, which could potentially make this a much more powerful and robust source.
Another outstanding question is whether this investment will result in observational data becoming more acceptable for coverage decision making. Currently, observational data are seldom used for making Medicare national coverage determinations. Some possible impediments to integrating observational data into coverage decisions may be a strong existing bias that randomized controlled trials (RCTs) are the only acceptable method for answering questions about relative effectiveness and an inadequate number of epidemiologists on staff who understand how to utilize observational data.
A Disconnect: The Lack of a Voice in Setting Priorities for CER Investment CER will be credible, timely, and relevant to Medicare beneficiaries only if CMS becomes a more active participant in defining research priorities. The agency needs comparative studies that address issues specific to the elderly, chronically ill, and disabled populations that it serves. When comparing the top 20 priorities established by CMS in 2007 with those established by the IOM and those established by the National Institutes of Health (NIH) for NIH Challenge Grants, 13 there are some notable differences (Table 1). Specifically, CMS was interested in looking at the impact of expensive cancer drugs, the use of reperfusion drugs for stroke prevention, and erythropoietin-stimulating agents and their use in cancer patients.
The Medicare program, either by design or by default, does not currently have an active voice in establishing priorities for CER investment. Although the legislation allocated space on PCORI's governing board for 2 federal or state representatives, 9 CMS is not represented. If CMS doesn't have a seat at the table in PCORI and if there's no effective mechanism for CMS to influence the types of studies being done, potentially there may be major gaps in the type of relevant and critical information that they need.
The Future of CER in the Medicare Program CER has the potential for broad usage throughout the Medicare program. Not only could the results be used to inform coverage decisions, but they could also be used to support patient decision aids and assist in coding decisions (determining whether there is enough of a significant therapeutic distinction for a particular product to assign it a new code). In addition, and perhaps even more importantly for CMS, CER data could be used in pricing-specifically in the area of developing policies for value-based insurance design and least-costly alternative pricing (or reference pricing). Through least-costly alternative pricing, Medicare contractors have the discretion to use relative costs in setting payments if there are 2 alternative items with equal efficacy. This has been used primarily to set payments for durable medical equipment. While a recent legal ruling has restricted use of this policy for drugs, this approach could help promote the generation of better CER for Medicare decisions. 14 For example, in the prostate cancer case, if research couldn't demonstrate a significant clinical difference between proton beam therapy and intensity modulated radiation therapy, then CMS could pay the same for those treatments rather than paying different amounts for each. Pearson and Bach suggest a framework where new technology would be paid for at a higher rate than existing technology for 3 years. At the end of the third year, cost for the new technology would be reduced to the existing technology cost unless the new technology demonstrated clinical advantages to justify the additional cost. 10 Another policy tool the agency has to help promote the development of CER is Coverage with Evidence Development (CED), also referred to as conditional coverage. CED links reimbursement to the requirement for prospective data collection. This policy tool allows CMS to help drive clinical research to ensure it is more relevant to its needs, such as ensuring that the

DISCLOSURES
This supplement was sponsored by PIM and StrataMed through an educational grant from Novo Nordisk. Penny Mohr received compensation from PIM for participating in the live continuing education activity on which this article is based and for writing the article. Mohr reports payment for contracted research with the National Pharmaceutical Council.

Author
population researched in the studies includes Medicare beneficiaries, and that the outcomes include significant clinical outcomes that are relevant to the beneficiaries. CMS has been involved in more than a dozen CED decisions over the last 15 years. 13 Although there was a brief hiatus in these types of decisions, activity has recently increased.
For many of the policies that are mentioned above, there are statutory ambiguities that keep the agency from being as proactive as they could be to use and promote the development of CER.

■■ Conclusions
Currently, the quality of existing evidence has often been a limitation for CMS in its decision making. CER has the potential to improve the quality of that evidence. However, there are some key challenges to address before CMS can use CER throughout the program. One challenge is a lack of sufficient staff and internal resources devoted to take advantage of CER. Another challenge to tying CER to reimbursement programs, such as least-costly alternative, is the long-standing institutional separation between the areas of coverage and payment policy, which would have to be overcome. In addition, statutory ambiguities inhibiting CMS's ability to use policies such as CED or least-costly alternative are limiting CMS's potential to be a value-based purchaser, and at least some of those statutory barriers must be corrected before the agency can use CER effectively.
In the end, CMS needs to be more effectively engaged in establishing research priorities to ensure that at least some of the CER data are relevant to their needs. CMS must find a way to make those needs transparent and become involved in determining research priorities. C omparative effectiveness research (CER) has the potential to improve decision making by helping those involved to understand who might benefit from what interventions and under what circumstances. It is important, however, to examine additional effects that CER may have on the health care industry so that we are not surprised by potential unintended consequences.
The pharmaceutical industry faces many requests for evidence and continues to strive towards producing the most comprehensive data possible. Today, the U.S. Food and Drug Administration (FDA) requires specific efficacy and safety data as part of the development and regulatory approval process. Payers want manufacturers to provide product dossiers with additional clinical, epidemiologic, and economic information.
With CER there will be increasing expectations that clinical trials will have active comparators, evaluate nonsurrogate endpoints (morbidity, mortality, hospitalizations), and measure cost. And payers, providers, and patients will want data generated from real-world environments. 1 This rising evidentiary bar will have implications, of which I will examine 3: (a) who has access to these new sets of data and who gets to talk about them, (b) how we interpret the results so that we don't confuse the average with the individual, and (c) whether this increase in CER data will have a positive or negative effect on innovation.

Restrictions on Dialogue
Historically, knowledge about a drug and its role in therapy was developed by manufacturers and shared primarily through dialogue between the drug manufacturer and payers, and with providers. In this environment, the manufacturer presents and shares data that were part of registrational trials, and payers might have some data of their own. Now, communication of evidence is changing as more parties participate in CER. In general there will be an ever increasing volume of data available on new technologies through CER and other real world database studies; however there are significant restrictions on how the manufacturer can communicate this information to stakeholders.
One of the regulations pertaining to how the pharmaceutical industry communicates with a payer audience is FDAMA 114. The majority (81%) of pharmaceutical industry experts (outcomes directors at major pharmaceutical and biotechnology companies) surveyed consider the use of FDAMA in their communications to payers when presenting health care economic information. However, almost as many (75%) also stated they would benefit from additional guidance by the FDA as to how such promotions should be made to payers. 2 A cornerstone of this new CER world will be the development of new databases. Two laws, the American Recovery and Reinvestment Act 3 and the Patient Protection and Affordable Care Act (PPACA), 4 Looking at CER from the Pharmaceutical Industry Perspective Robert W. Dubois, MD, PhD ABSTRACT BaCkgRound: Comparative effectiveness research (CER) is increasing as an element of health care reform in the united States. By comparing drugs against other drugs or other therapies instead of just to placebo, CER has the potential to improve decisions about the appropriate treatment for patients. But the growth of CER also brings an array of questions and decisions for purchasers and policy makers that will not be easy to answer and which require significant dialogue to fully understand and address. oBjECtIvE: to describe some of the impact, both positive and negative, that comparative effectiveness research (CER) may have on the pharmaceutical industry.
SuMMaRy: as CER data proliferate, questions are being raised about who can access the data, who can discuss it, and in what forums. Regulations place different communication restrictions on the pharmaceutical industry than on other health care stakeholders, which creates a potential inequality. another CER consideration will be the tendency to apply average results to individuals, even if not every individual experiences the average result. Policy makers should implement CER findings carefully with a goal toward accommodating flexibility. a final impact to consider is whether greater expectations for CER will have a negative or positive effect on incentives for drug innovation. In some cases, CER may increase development costs or decrease market size. In other cases, better targeting of trial populations could result in lower development costs.

What is already known about this subject
• This article invites dialogue about the communication restrictions between the pharmaceutical industry and other health care stakeholders, with an eye toward affecting future regulations in a positive way. • As policy makers expand their use of CER data, they should be cautious in applying average results to subgroups or individual patients. • CER's effects on future innovation may be both negative (e.g., increased trial sample sizes, longer-term endpoints, or the risk of unfavorable results) and positive (e.g., reduced sample sizes by identifying target subgroups earlier in development, or real-world data that show increased value for the drug, spurring further development). Therefore, regulators and developers must begin exploring ways to prepare for potential impacts.

Applying Average Outcomes to Individual Patients
Every day, peer-reviewed publications or communication from the National Institutes of Health or the Agency for Healthcare Research and Quality detail the latest results of new clinical studies or systematic reviews. In the future, it will be overwhelming to try to keep up with the new data from the hundreds of studies that will be generated every year. To simplify information overload, the inclination is to remember the bottom line-is A better than B? Although we know logically that individual patients respond differently, we may focus upon the average result. But the reality, of course, is that the average is a composite and may not apply to each individual. As Kravitz et al. (2004) point out, heterogeneity of treatment has many meanings. 6 Inter-study heterogeneity entails comparison of results among several studies. Did those separate studies lead to very similar or very different results? Perhaps the population differed in each study and accounts for the different results. A second type of heterogeneity (inter-patient) relates to typical comparisons of A versus B where the data show that some patients did well and others did not. Rose's 1985 epidemiology paper is a well-known source on this topic. 7 The issue of heterogeneity of treatment has current relevance as the draft of the Essential Health Benefits (a component of the PPACA) requires that only a single agent be available in each drug class. 4 Perhaps the most clinical-or policy-relevant type of heterogeneity relates to how patients' individual responses may differ from the study average.
Let's examine a hypothetical example where patients on average do better on the "blue pill" than on the "red pill," an example similar to one recently used by President Obama in a national broadcast. 8 But individual patients may do better on the red pill than the blue pill for many reasons, including biologic or demographic distinctions or patient preferences. Policy makers reading an article about a study may note that the average result shows blue is better than red, and develop policies stating that everyone with a particular condition should receive the blue pill. The problem, of course, is that developing such a policy does not account for those individuals for whom the red pill works better. How can decision makers in managed care and PBMs work around this problem?
One commonly discussed theme is subgroup analysis. For example, in this same hypothetical study, we may discover that younger people respond differently than older people. We could falsely assume that the heterogeneity is now resolved and recommend that all younger people receive the blue pill, and all older people receive the red pill. 9 Unfortunately, the provided funding to invest in the development of various sources of information. For example, the FDA will have databases from which it will be mining and releasing information. Medicare will be making some of its data available for comparative purposes. Various other public and private databases will be used for CER. All of these information repositories will allow more and more researchers-from manufacturers, academics, the government, health plans, and others-to compare the performance of drugs to each other and to other therapies in ways that previously have not been feasible.
With researchers and others looking at available data, questions arise as to who has the right to access this information, and who has the right to talk about what is found. Historically, it has been very difficult and expensive to access data, and those data were often limited in the populations they contained. But, as more data opportunities become available, there needs to be an open debate about the pros and cons of having this information widely available. The more people who have access to the data and can work with it, the more this issue has relevance. Multiple researchers will be able to analyze the same data and openly discuss different results, methods, or endpoints. Will pharmacy benefit management (PBM) companies or health plans consider supporting these open databases and sharing their own information? Will all parties have similar access rights, or will just selected government personnel or a handful of researchers have that access? Or will access to taxpayer-funded data be generally available? There may be no right answer, but the conversation should begin.
Another aspect of communication relates to who can communicate CER findings. Currently, the pharmaceutical industry can speak to physicians and pharmacists about their drugs as long as the conversation addresses using the drug for on-label indications. They can describe the indications and the endpoints that were part of registrational trials. Off-label communication is far more restricted even though off-label use occurs quite commonly, especially in oncology. 5 Other organizations or agencies do not have these restrictions. Moreover, when these groups conduct a study and publish the results, should the pharmaceutical companies that created the drugs be allowed to comment on the studies, whether to agree or suggest alterations or provide additional data? Currently, rules place far greater communication restrictions on the pharmaceutical industry than on anyone else in health care. This limits the pharmaceutical industry from responding with relevant information to the public debate. Today, pharmaceutical companies need multiple clinical trials to support assertions, but all other organizations-payers, the government, and the FDA-can make comparisons without conducting multiple trials or obtaining approval prior to dissemination. In other words, the dialogue becomes unequally restrictive, potentially to the detriment of the patients we aim to serve.
As CER increases the pool of information to discuss, the restrictions in place today become increasingly problematic. Another potential consequence is that certain drugs may not be pursued for clinical development. If a drug comes to market for indication A, but its clinical effectiveness is not shown to greatly differ from the alternative, payers may decide not to reimburse it or to place this new drug in a higher tier. If that happens, the manufacturer may not continue drug development-and risk-of testing it for indication B, where the drug might have performed better than any other drug available. And if a drug never makes it to the market, its descendent, derivative drugs will not be discovered. This is an important issue as a premium price cannot be obtained for a truly innovative product if the incremental steps to its development are not met. However the payer is not willing to pay for incremental developments. If this is the case then these costs get absorbed into the overall development of the innovative product, as recently described by Mansley,et al. 11 On the other hand, CER could have a positive effect on innovation. For example, a manufacturer may identify a patient subgroup with biologic or genetic tests, perhaps, before starting clinical trials. In that case, the trials may show clear-cut success, and the clinical trial could be small, keeping costs low. In addition, as a manufacturer develops CER spanning multiple outcomes, including long-term real-world data, productivity improvement, and quality of life, perhaps the research will find even more value.
The increasing expectations of CER present many challenges. Patients and other health care decision makers want this information, and it is correct and beneficial to ask for it. However, it is important to recognize that there are consequences that may affect the viability of bringing drugs to market in both negative and positive ways. As Berger and Grainger (2010) point out, it is clear that there will be ongoing debate on how the pharmaceutical industry should incorporate CER into drug development. 12 Organizations will need to consider how they will balance the impact of CER's impact on innovation against the benefits of CER evidence.
The National Pharmaceutical Council has begun modeling projects to evaluate some of the impacts CER may have on the future of the pharmaceutical industry, but it is too soon to share data or make any predictions. It is not too soon, however, to consider the questions and begin the necessary dialogue toward solutions. And it is likely that there will be solutions, because the value of CER is not, in this author's opinion, in question. It is how to manage the impacts of CER that requires our attention. One possible example might be to offer different durations of patent life depending on whether a manufacturer conducts placebo trials or the longer and more expensive (and more desirable) comparative trials.

■■ Conclusions
It is important for the pharmaceutical industry to participate in the dialogue on the way CER is conducted, interpreted, and implemented. Many other countries are further along than the reality is that subgroups will not fully solve the problem, because there will still be heterogeneity even within these subgroups. In other words, not all of the older people will do better on the red pill. Using subgroups may be a step in the right direction, but it is not a complete solution.
CER data should affect population reimbursement policies, but when is that safe and when is it problematic? If patients have a chronic, slowly deteriorating or nondeteriorating illness (such as hay fever or fibromyalgia), they can safely initiate treatment with the blue pill, and if that does not work, they can try the red pill next. On the other hand, if they have a severe infection or cancer that is rapidly progressive, they do not have the luxury of trying one option and then switching to another. As policymakers integrate the red and blue pills into their reimbursement approaches, they must keep in mind that in some circumstances it will be fine to institute policies where everyone first should receive the blue pill, or the blue pill merits a lower copayment or a different tier, and then anyone who fails on the blue pill can move on to the red pill. But in other circumstances where perhaps the situation is clinically different, the risks are different, the deterioration possibility is different, or the way the curves overlap is different, then having a strict, centralized policy may be the wrong solution. In those circumstances, it might be better to grant more freedom to the doctor and patient, creating a decentralized decisionmaking policy.
Patient cost-sharing should also consider treatment heterogeneity. If someone is biologically different and he has tried the blue pill because that is the policy, but he does not respond to it and therefore has to try the red pill, why is his cost-sharing higher? It may be a nonpreferred drug, but biologically he failed the preferred drug. If he is biologically different, should he be penalized by having to pay more?
Integrating CER into decision making will bring these questions to the forefront. With comparative data, policy decisions should not be treated as one-size-fits-all. Sometimes centralized reimbursement approaches make sense. It is cost-effective, it is the right thing to do, and it will not harm patients. But other clinical circumstances may call for more flexible policies that allow the doctor and patient more ability to choose.

How Will the Growth of CER Impact Innovation?
The growing demand for CER will impact the pharmaceutical industry in many ways, and its effect on innovation may be difficult to predict. In a CER environment, the costs of drug development may rise. For example, if the marketplace requires active comparators instead of placebo, the sample size required to show statistical differences will increase. A larger sample size entails greater cost. 10 Second, demands for long-term clinical rather than short-term surrogate endpoints will increase study duration and attendant costs. Third, CER entails exploration of response in patient subgroups. To achieve adequate power, sample sizes will need to be larger. If particular subgroups show greatest benefit and lesser harm, then patients gain from this new knowledge, but the market size and revenue to offset United States in the application of CER to health technology decisions, and the pharmaceutical industry can benefit from internal discussions across global sectors in how this is done, especially since multinational corporations already are dealing with CER in its various forms in other parts of the world. 13 There should be broad access to taxpayer-funded data by all stakeholders, and the public and pharmaceutical industry should have the right to talk about whatever CER studies are performed. We all need to explore when average results may or may not be applied to individual patients. An early and active public dialogue will benefit all stakeholders.

DISCLOSURES
This supplement was sponsored by PIM and StrataMed through an educational grant from Novo Nordisk. Robert Dubois is an employee of the National Pharmaceutical Council, which is a member-sponsored organization of the pharmaceutical industry. NPC is a health policy organization and does not lobby.
S ince 2009, health care reform has become one of the driving forces behind comparative effectiveness research (CER). Health care reform has also driven other changes in the marketplace, such as additional coverage being added for different treatments, and coverage being extended to people who weren't eligible for care previously. Clearly, we will continue to see changes in the market over the coming years. The challenge is in predicting how consumers and employers will react to those changes, how CER will affect the market, and how managed care organizations (MCOs) will respond to the increase in information.
Before we can understand the role of CER in a reformed market, we need to have some idea of what that market will look like and what the market will want. Figure 1 illustrates a 2009 Kaiser Commission on Medicaid and the Uninsured/ Urban Institute analysis showing where U.S. adults aged 19 to 64 years currently obtain health care coverage. The analysis shows 23% of this population is uninsured, 59% obtain coverage through their employer, and 6% are in the private market (nongroup). Medicaid and other public payers make up the remaining 13%. 1 By 2019 up to 32 million people will gain coverage who have not had coverage before, including large increases in the Medicaid population. The exchanges will enroll previously uninsured individuals, individuals who will lose employerbased coverage or who cannot afford employer-based coverage due to increasing costs, individuals who would otherwise have purchased health insurance in the nongroup private market, and adults above the 138% federal poverty level (FPL) who will lose their Medicaid coverage. 2 As shown in Figure 2, these estimates indicate that from 2009 to 2019, we will see an increase from 19% to almost 40% of the population making health care decisions who previously were not in the employer market (Medicaid, other public groups, and private nongroups). Another aspect of the changing market is that more groups are giving individuals a choice, even within employer-based plans, where choice may have been more limited in the past.
All of this means we must carefully consider what the consumer is going to want. At SelectHealth, we are researching and talking to patients in our market, trying to better understand what consumers want. Obviously consumers want quality products. But health insurance can be a very confusing marketplace, so they want products that are simple to understand and use. A large part of the population wants low cost and another portion of the population is willing to spend extra for value. To serve those customers, companies willing to invest in CER may be able to provide both low-cost and high-value options. Finally, will consumers be willing to compromise on the amount and type of choices they're offered?
Analyzing what employers want in health care reform reveals 3 clear goals. Employers want to contain costs, encourage healthy lifestyles, and improve the quality of care. 3 Mushlin and Ghomrawi (2010) looked at the components of health care reform that may help the health care system Looking at CER from the Managed Care organization Perspective

H. Eric Cannon, PharmD, FAMCP ABSTRACT
BaCkgRound: the amount of available comparative effectiveness research (CER) is increasing, giving managed care organizations (MCos) more information to use in decision making. However, MCos may not be prepared to integrate this new and voluminous data into their current practices and policies.
oBjECtIvES: to describe ways that health care reform will affect MCo populations in the future, to examine examples of how MCos have utilized CER data in the past, and to identify questions that MCos will have to address as they integrate CER into future decision making.
SuMMaRy: unquestionably, health care reform will change the u.S. market. Millions more insured individuals will be making purchasing decisions. In addition, health care reform will mean more CER data will be available, affecting the decisions MCos must make. In the past, MCos may not have used CER as effectively as they could in making formulary and other policy decisions. However, there are examples that show how CER can be integrated effectively, such as Intermountain Healthcare's use of CER to create treatment guidelines, which have been shown to lower costs and improve delivery of care. In the future, MCos will need to assess their own abilities to utilize CER, including their infrastructure of expertise, hardware, software, and protocols and processes. MCos will also need to understand how pertinent CER is to their own needs, how it may affect benefit design, and how it will affect their customers' needs.
ConCLuSIon: Health care reform, and the resultant growth of CER, will have significant impact on MCos, who will need to invest in better infrastructure and new understandings of a transforming market, changing customer bases, and evolving data. • Under health care reform, millions more insured individuals will be making purchasing decisions. • In the past, managed care organizations (MCOs) may not have used comparative effectiveness research (CER) as effectively as they could in making formulary and other policy decisions. • Traditionally, MCOs recognized facilities, hospitals, and physicians as stakeholders and involved them in decision-making discussions. Seldom, if ever, have MCOs involved the patient in that process.

What is already known about this subject
• This article suggests ways in which CER data can be used more effectively, such as in creating treatment guidelines and pathways for providers. • MCOs will need to assess their own abilities to utilize CER, including their infrastructure of expertise, hardware, software, protocols, and processes. • To fully utilize CER, MCOs will need to understand how pertinent the available CER is to their own needs, how it may affect benefit design, and how it will affect their customers' needs. • MCOs must participate in CER discussions and find ways to encourage CER that will produce results useful for decision making. become more efficient and more effective. 4 They wrote that financial incentives, programmatic initiatives, and organizational changes could be considered "blunt instruments" that may nudge the health care system toward efficiency. On the other hand, they considered CER a "sharper tool" that could inform change and guarantee the enhanced health of the population by delivering comparative and precise information. The Evidence-based Practice Centers (EPCs) funded by AHRQ have developed a system of grading the strength of a body of evidence when comparing medical interventions. The goal is to provide an objective assessment of the strength of evidence that decision makers can use in making their assessments. Four domains are required and include risk of bias, consistency, directness and precision. 5 Additional domains can be included as appropriate including dose response association, confounders, strength of association and publication bias. An overall strength of evidence grade is given as either high, moderate, low, or insufficient.
To see how CER can be used as a sharper tool in the future, it may help to look at an example of how we've used CER data in the past.

How Have We Used CER with Diabetes Medications in the Past?
A 650-page report first published by the Agency for Healthcare Research and Quality (AHRQ) in 2007 and updated in 2010 analyzed CER data for oral diabetes medications for adults with type 2 diabetes. The report showed that most diabetes medications (metformin, thiazolidinediones, sulfonylureas, and repaglinide) reduce hemoglobin A1c by a similar amount (about 1%). 6 The report found that metformin as monotherapy was more effective at reducing A1c than were DPP-4 inhibitors as monotherapy. Most metformin combinations were more effective than metformin monotherapy. The report also found a limited ability to draw conclusions on the glucagon-like peptide (GLP)-1 analogs and agonists because the evidence was graded as either low or insufficient. Researchers found that metformin consistently promoted weight loss, acarbose promoted weight loss or was weight neutral depending on the study, and low-density lipoprotein (LDL) cholesterol levels decreased with metformin or second-generation sulfonylueras, along with many other findings.
Another AHRQ report from 2008 reviewed insulin analogues, showing that premixed insulin analogues are better at lowering both A1c values and postprandial glucose than longacting insulin analogues. 7 But, the long-acting analogues cause less weight gain and less hypoglycemia. Comparing premixed insulin analogues with noninsulin medications, 3 categories showed moderate strength of evidence indicating that the premixed were probably better, while in 2 categories the noninsulin medications performed better. 7 Each of these characteristics form part of the therapy decision that should be incorporated when treating diabetes in an individual patient.
But, have we really used these data to inform our formulary decisions? An analysis of SDI's Spring 2009 Managed Care Formulary Drug Audit could be expected to show that HMOs' placement of subsets of antidiabetic agents in preferred positioning might vary based on the available CER data. For  Looking at CER from the Managed Care Organization Perspective example, the data showed that pioglitazone raises triglycerides less than rosiglitazone. 7 But in looking at where health plans across the country have positioned those, they were almost equal in 2008 (pioglitazone placed in preferred positioning by 100% of HMOs, and rosiglitazone placed in preferred positioning by 98% of HMOs). 8 Discrepancies such as these point out that managed-care plans may not be doing all we can to use the CER data and guidelines in meaningful ways when it comes to positioning products within a formulary. Despite this, there are examples of how MCOs have used CER data to drive how guidelines are established.
One of the predominant ways Intermountain Healthcare has used available CER data is in the development of disease management programs, called clinical programs, that provide tools and information to help practitioners deliver care in a consistent and integrated way. Practitioners must have access to the data and understand how to apply that data operationally, so Intermountain has designed clinical programs to guide and define new disease management systems and integrate them into routine care throughout the Intermountain system. Each disease management system includes an evidence-based care process model, patient education materials, clinical support materials to make care delivery easier, and a data measurement and reporting process to analyze practice patterns. Reducing variability in the process of care allows assignment of outcomes to a causal treatment variable. In other words, Intermountain's main goal is to reduce the variation within treatment, thereby producing better outcomes.
While Intermountain's formulary may not vary much from other MCOs as far as positioning, CER data have been incorporated in developing Care Process Models (CPMs). CPMs are evidence-based guidelines for common and chronic conditions. The CPM teams have tried to ensure all the products are available within the formulary, and then they have taken it a step further within the clinical pathways to define exactly where each product might be used. For example, in the treatment of diabetes, the CPM recommends starting with metformin. If control is not obtained then another agent can be added, depending on the specific needs of a patient. Sulfonylureas or insulin might be added as low-cost options. In a patient who needs some weight loss, the options might include a DPP-4 inhibitor or a GLP-1 agonist. 9 Can the use of CER data lead to high-quality, cost-effective care in the treatment of diabetes? Intermountain has proven that its diabetes clinical program is associated with improved performance in diabetes clinical measures. 10 Intermountain CPMs have helped control costs and improve care.

How Will CER Affect MCOs?
Every MCO will need to conduct its own introspective review and ask what it must do to be in a position to use the growing volume of CER information. At every level, individuals need to be educated, understand how the research was done, determine how it applies to their population, and identify how it applies in different demographics. Taking it further, we must also ask: where have we used comparative effectiveness in the past, where would we like to use it in the future, and what will it take to accomplish that? Some of these questions will be difficult, with implications that will need to be managed carefully. For example, if observational data tells us a particular therapy doesn't work as well as others in certain populations, how do we handle patients who are already using that therapy successfully?
As more individual consumers begin making purchasing decisions, we'll have to ensure that those individuals are choosing our organizations and putting their money and their support behind us, or we won't remain viable. So how will we incorporate those patients and individuals into our decision making? In the past, our stakeholders clearly have included facilities, hospitals, and physicians, and we as MCOs have involved them in these discussions and in making decisions. Seldom, if ever, have we involved the patient in that process. But in a reformed market, the individual will play a greater role in health care decisions and will have a greater need to be engaged. How do we disseminate more information to them? The Institute of Medicine recommends that consumers should have involvement in CER. Kreis et al. (2012) surveyed 17 organizations conducting or commissioning systematic reviews and found that 7 of them involve consumers at a programmatic level, through one-time consultation or on ongoing collaboration. 11 They conclude that an assessment of which approaches are most effective is required to further define the most appropriate involvement of consumers in CER.
At a basic level, we need to work at building the infrastructure to incorporate CER. Within each of our organizations, we will need to develop the expertise, the hardware, the software, and the protocols to use this new and growing volume of data. The Center for Medical Technology Policy held a CER summit in November 2010, and one of the issues that resulted was the need for consistent data standards in data generation and utilization. 12 The availability of data allowing comparisons is critical to CER. Hirsch et al. (2011) describes examples of the data infrastructure needed to support various CER methods, some of which are already slowly developing. 12 For example, to support meta-analysis and guideline development, we will need standardized data collection in clinical trials to allow comparisons across trials and practices, and strengthened national registries that incorporate data from clinical practice. The integration of patient-reported outcomes will require development of patient-accessible platforms in which data can be directly entered and integrated into electronic health records (EHRs), as well as interfaces that allow clinicians and researchers to track data in real time. And coverage with evidence development will require the development of integrated EHRs that allow a range of outcomes to be assessed, and expedited analysis in clinical trials by relying less on manual aggregation of data. 12 As we design the infrastructure and operational systems to adjudicate the claims made from CER analyses, how does that affect benefit design? Will there still be a role for an open formulary or are we going to be looking at more closed formularies based on the evidence?

■■ Conclusions
As we evaluate CER and its many impacts on the U.S. health care system, it's clear that many different stakeholders will have different agendas and needs for the data that will be generated and analyzed. Several years ago, Sean Tunis, MD, discussed how the funding for CER could affect the ability of MCOs to use that research. He said "If the money goes through the usual channels and is primarily controlled by the academic community and the existing research infrastructure, it's going to be a large volume of answering questions that don't matter to decision makers." 13 Several health plans across the country have recognized the need for more involvement and are starting find ways to create collaboration. For example in the fall of 2011, Pfizer and Humana announced a partnership to improve health care delivery to seniors. Researchers and health care experts from both organizations will be brought together to study key issues and deliver interventions to reduce inefficiencies in the management of chronic conditions such as pain, cardiovascular disease and Alzheimer's disease. 14 From a managed care perspective, the key as we start doing more and more CER is to ensure we are asking questions that are relevant to us as MCOs and are producing results that we can actually use to make decisions. If it isn't relevant to us, it really won't provide any value for us.

DISCLOSURES
This supplement was sponsored by PIM and StrataMed through an educational grant from Novo Nordisk. Eric Cannon received compensation from PIM for participating in the live continuing education activity on which this article is based and for writing the article. Cannon reports a consulting relationship with Novo Nordisk.